UE4 Lightmass for Large Console Games (UE4 Lightmass Deep Dive)

Lightmass Operation for Large Console Games © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Osamu Saito SQUARE ENIX CO., LTD. Environment Designer Developing games full of hopes and dreams © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We encountered various issues in the following 3 areas Large Console Game © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Note: This is not a “Success Story” but rather a discussion of “ongoing” issues. Because our team is still struggling to solve these issues, our workflow may change in the future. If you know of better approaches or methods, Please let me know (right after this session). © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Let’s start with Game © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

© 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Recreation of a Lounge room © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Unreal Engine4 Photo © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Unreal Engine4 Does it seem like we got the lighting right? Photo © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Bad Lighting Unreal Engine4 Photo © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What’s the problem? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Let’s add a moving object and you will see. © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

© 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What is the cause? ? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

PLV Precomputed Lighting Volume © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

These black dots © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

PLV(Precomputed Lighting Volume) is indirect lighting data which is generated in the space when using Lightmass © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

PLV is used to light dynamic objects. © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Errors in PLV cause strange Lighting © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Since most games use dynamic meshes You should pay attention to PLV when building lighting © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This is the cause of the lighting error in the previous slide © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Our team’s solution © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Re-examine lighting method Re-examine sampling method © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Re-examine lighting method Re-examine sampling method © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We reproduced the lighting error with static lights © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Static Light and Emissive Light are the most likely causes of errors in PLV © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Static Light Direct light stored in PLV Emissive Light Very poor PLV accuracy © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Therefore Static Light Direct light stored in PLV Emissive Light Very poor PLV accuracy © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Our team decided to limit their usage Static Light Direct light stored in PLV Emissive Light Very poor PLV accuracy © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We decided to prohibit the use of Both static light and Emissive light with dynamic meshes. © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This is unfortunate, But we have no choice © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Re-examine lighting method Re-examine sampling method © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

ILCQ Settings © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What is ILC? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

ILC Indirect Lighting Cache Indirect lighting caches are created for each dynamic mesh by referencing the PLV © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

ILCQ Indirect Lighting Cache Quality Point Volume Sampling: one point (center of bounding box) Sampling: 5×5×5 grid (equal to bounding box size) © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

ILCQ Indirect Lighting Cache Quality Point Volume Sampling: one point (center of bounding box) Sampling: 5×5×5 grid (equal to bounding box size) © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

ILCQ set to “Point” causes errors © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

But ILCQ set to “Volume” is ok © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

© 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

As a result, draw cost goes up © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

As PLV is referenced more, the draw cost increases so Be careful when using “Volume” for large, fast-moving objects! © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Let’s go back to the previous PLV error © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Static Light Emissive Light Stopped direct light from being stored in PLV (our customization) Improved PLV accuracy (From UE4.13 onwards, PLV uses Adaptive Sampling) © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Let’s move on to the next topic © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Console © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Console Development always involves strict limitations © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

For example, Memory © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

1GB Texture pool of Unreal Engine is 1GB by default © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Equal to 4K texture x 50 sheets © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Out of this, how much memory can actually be used for lighting? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

200 MB (1/5) About 10 4K sheets © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

For this much area? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

No. For this much area! © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Our development team’s workaround © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Only “BLUE” is allowed for Lightmap Density © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

How much resolution does Lightmap Density “BLUE” represent? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

1m ≈ 5 pixels © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

In extreme cases we allow “GREEN” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

1m ≈ 20 pixels © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

How about “RED”? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

No way! 1m ≈ 80 Pixels © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We ran into this... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Completely red map was checked in due to incorrect settings © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

4GB in Lightmaps alone © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Too big to be stored in texture pool Failed to visualize due to error © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Overly large files caused server access errors © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This upset some people... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

1GB If uploaded file size exceeds 1 GB, our system would send a warning e-mail © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

So... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What if we use “BLUE”? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

So many Lightmap errors © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Intersection errors / UV seam errors Padding errors / UV grid errors © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Creating appropriate Lightmap UVs helps to reduce errors © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We had to compromise to some extent © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Folige lightmap error © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Column Lightmap error Intersection error © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

If you know of better solutions, please let us know. We welcome your advice or suggestions. © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

One more difficult topic © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Though we said 200MB for this scale... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We actually need to reduce it more © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Assuming we have 30 maps at this size... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Lightmaps alone are 200MB×30 = 6GB © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This accounts for more than 1/4 of a Blu-ray disk © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

That’s too much © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Today’s last topc © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Large © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

In my opinion, this is the most difficult issue © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Large project = A large number of people involved © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Large project = A large number of people involved = simultaneous work © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Large project = A large number of people involved = simultaneous work = Number of people working on lighting goes into double digits © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Various methods Different preferences © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

For consistency, We needed a standard © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We chose “Illuminance-based Lighting” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What is Illuminance? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

“In photometry, illuminance is the total luminous flux incident on a surface, per unit area. It is a measure of how much the incident light illuminates the surface, wavelength-weighted by the luminosity function to correlate with human brightness perception. Similarly, luminous emittance is the luminous flux per unit area emitted from a surface. Luminous emittance is also known as luminous exitance.” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

“In SI derived units these are measured in lux (lx) or lumens per square metre (cd·sr·m−2). In the CGS system, the unit of illuminance is the phot, which is equal to 10000 lux. The foot-candle is a non-metric unit of illuminance that is used in photography. Illuminance was formerly often called brightness, but this leads to confusion with other uses of the word, such as to mean luminance. "Brightness" should never be used for quantitative description, but only for nonquantitative references to physiological sensations and perceptions of light. The human eye is capable of seeing somewhat more than a 2 trillionfold range: The presence of white objects is somewhat discernible under starlight, at 5×10−5 lux, while at the bright end, it is possible to read large text at 108 lux, or about 1000 times that of direct sunlight, although this can be very uncomfortable and cause longlasting afterimages.” Excerpt from Wikipedia https://en.wikipedia.org/wiki/Illuminance © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

“It is an index literally for measure of how much the incident light illuminates the object and used to indicate brightness. If the lit object is not vertical to the ray as shown in right image, illuminance of the lit object decreases, so in order to interporate this. The difference between illuminance and light flux is how to consider the dimension of the lit object. For example, a book which is closer is darker than the book which is far away. This phenomane is reffere to as low illuminance. This is why lux is used to indicate how much the light illuminates the surface of a desk While candela is used as a unit for an electric light which is a light source. The brightness per unit perceived by eyes does not depend on distance but is directly proportional to illuminance of the object. However, the brightness per unit perceived by eyes attenuates as the object gets further if dimension is too small even for the object with same dimension and illuminance. Extraction form Wikipedia https: //ja.wikipedia.org/wiki/%E7%85%A7%E5%BA%A6 © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

In other words, “Brightness” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Sunny 100,000 Lux Cloudy 5,000 Lux Office 500 Lux Underground Parking 30 Lux Street light Moon light 0.1 Lux 5 Lux Illuminance is a unit of human “brightness perception” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Why use “Illuminance”? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We were often asked: “What is the sun’s intensity in terms of a directional light?” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

.... I have no idea © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

How could we find the answer to this? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This brought us to “Illuminance” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Unknown value Lumens Looks like lumens but actually unknown value Unknown value UE4 uses various intensity standards for the 4 respective light types © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

“Illuminance” is not affected by the type of light source because it is a unit for received light © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Using “Illuminance”, the intensity of the sun can be calculated for either a Point light or Directional light © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Using “Illuminance” allows lighting in one standard regardless of light type or material © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Now let’s calculate the brightness of the sun in terms of Directional light intensity © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This part might be a bit boring but... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

First, create a perfectly diffuse surface by placing a plane with the following settings: BaseColor＝1.0 Roughness=1.0 Metal=0.0 Specular=0.0 © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Then place a Point light 1m above the surface © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Increase intensity until the L value in HDRVisualize mode reaches 1.0 © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

At this point, the Lumen value of the Point light is about 1966 lm 1966/4π ≈ 156.45lx L value of 1.0 is equal to illuminance of about 156 lx © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Next, place a Directional light pointing directly downwards © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We get an L value of 1.0 when the intensity is about 3.15 So a Directional light with intensity of 3.15 corresponds to Illluminance of about 156 lx © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Sunny Cloudy 100,000 Lux 5,000 Lux Office 500 Lux Underground Street light Moon light Parking 0.1 Lux 5 Lux 30 Lux Illuminance on a sunny day is 100,000 Lux © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

100000/156*3.14 ≈ 2012.8 is the Illuminance value of a Directional Light for a sunny day © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

2012.8 © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Adjust Exposure as necessary © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Unknown value Lumen Looks like lumens, but Unknown value Illuminance Unknown value “illuminance” is a standard which can be applied to different light types © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Using “Illuminance” as a standard makes it easier to use the real world as a reference © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Our team measured illuminance and IBL at regular intervals from sunrise to high noon © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We gathered data on Sun elevation, sun elevation angle, color temperature, light volume, and IBL © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Lighting environment of each time period was reproduced on UE4 and checked for accuracy... © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

We built a standardized light rig © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

This allowed us to standardize the various lighting methods and approaches that were being used © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Initial lighting design cost was reduced © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

However © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Using “Illuminance” as a standard caused some errors © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

16 bit limitation © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

UE4 GBuffer is limited to 16 bits per channel © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Maximum value of 16 bit is “65535” © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What happens if we use our previous 2012.8 value Directional Light to light a sphere with Roughness set at 0.0? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Infinity! Scene color brightness is blown out at the highlight point shown above © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Conversely, with the value set to 0.001, debug view and gizmo display are blown out © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

“Illuminance” as a standard = “Reality” as a standard © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

HDR data of the real world is too big for 16 bit © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

What should we do? © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Are there better lighting workflows? Please let us know. We are happy to hear your suggestions. © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Wrap up © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Game Watch out for lighting on dynamic objects! Save us, Shinoyama-san! CS Large Limitations are very strict Somebody please help us with light bake errors! We are proud of creating our Illuminance standard! But the Gizmo burns our eyes ;) © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Thank you for listening to our presentation © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.

Please feel free to ask questions UnrealEngine4 is a trade mark or a registered trade mark of Epic Games. PARAGON is a trade mark or a registered trade mark of Epic Games. Blu-ray is a trade mark or a registered trade mark of Blu-ray Disc Association. Any other company names and product names used in this document are trade marks or registered trade marks of the individual companies. © 2016 SQUARE ENIX CO., LTD. All Rights Reserved.